Papers with LLM representations
Rhetorical Questions in LLM Representations: A Linear Probing Study (2026.acl-long)
Copied to clipboard
| Challenge: | Rhetorical questions are asked not to seek information, but to persuade or signal stance . how large language models internally represent rhetorical questions remains unclear . |
| Approach: | They analyze rhetorical questions in LLM representations using linear probes on two social-media datasets with different discourse contexts. |
| Outcome: | The results show that rhetorical signals emerge early and are most stably captured by last-token representations. |
EffEval: A Comprehensive Evaluation of Efficiency for MT Evaluation Metrics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent surge of interest in developing evaluation metrics based on pretrained large language models (LLMs) can better cope with lexical variation. |
| Approach: | They propose to replace computation-intensive transformers with lighter alternatives and employ linear and quadratic approximations for alignment algorithms on top of LLM representations. |
| Outcome: | The proposed approach replaces computation-intensive transformers with lighter alternatives and employs linear and quadratic approximations for alignment algorithms on top of LLM representations. |
SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Models (2026.eacl-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved remarkable progress, yet their internal mechanisms remain largely opaque. |
| Approach: | They propose an agent-based framework that recasts feature interpretation from a passive, single-pass generation task into an explanation-driven process. |
| Outcome: | The proposed framework produces explanations with significantly higher generative and predictive accuracy compared to state-of-the-art baselines. |
GrEmLIn: A Repository of Green Baseline Embeddings for 87 Low-Resource Languages Injected with Multilingual Graph Knowledge (2025.findings-naacl)
Copied to clipboard
| Challenge: | Contextualized word embeddings are available for many languages, but their coverage is limited for low resourced languages. |
| Approach: | They propose a method that integrates multilingual graph knowledge into the embeddings to make them green. |
| Outcome: | The proposed method outperforms state-of-the-art embeddings on lexical similarity task while being parameter-free at inference time. |
Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspective (2025.naacl-long)
Copied to clipboard
Shenglai Zeng, Jiankun Zhang, Bingheng Li, Yuping Lin, Tianqi Zheng, Dante Everaert, Hanqing Lu, Hui Liu, Hui Liu, Yue Xing, Monica Xiao Cheng, Jiliang Tang
| Challenge: | Existing studies have shown that LLMs struggle to identify the boundaries of their own knowledge and tend to prioritize external information over internal knowledge learned during pre-training. |
| Approach: | They conduct a comprehensive analysis of LLM representation behaviors and demonstrate the significance of using representations in knowledge checking. |
| Outcome: | The proposed classifiers improve performance even when dealing with noisy knowledge databases. |
From ID to LLM: Rethinking Representation Learning for Recommendation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies indicate a fundamental incompatibility between ID representations and language model (LM) representations as they capture behavioral and semantic spaces respectively. |
| Approach: | They propose a Profile-then-Embedding framework for recommendation that integrates semantic user and item profiles and a Personalized Embedded stage to encode these profiles into task-aligned recommendation embeddings. |
| Outcome: | The proposed framework achieves significant gains across three benchmark datasets, including cold-start and long-tail scenarios. |
Representing LLMs in Prompt Semantic Task Space (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) achieve impressive results over various tasks, and public repositories contain an abundance of pre-trained models. |
| Approach: | They propose an efficient, training-free approach to representing LLMs as linear operators within the prompts’ semantic task space. |
| Outcome: | The proposed representations achieve state-of-the-art results on success prediction and model selection tasks with notable performance in out-of sample scenarios. |
Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completion (2025.findings-acl)
Copied to clipboard
| Challenge: | Using later reasoning steps does not always improve classification, suggesting LLMs encode key information early. |
| Approach: | They propose a method to predict the success of a zero-shot Chain-of-Thought process by using LLM representations that are based on initial steps representations. |
| Outcome: | The proposed method performs well even before a single token is generated, suggesting that crucial information about the reasoning process is already present in the initial steps representations. |
Modeling Multi-Dimensional Cognitive States in Large Language Models under Cognitive Crowding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) mainly address isolated tasks such as emotion analysis or stance detection. |
| Approach: | They propose a large-scale model that combines large-level annotations with hyperbolic space to model human cognitive states. |
| Outcome: | The proposed model outperforms baseline models on cognitive dimensions on single dimension tasks while retaining strong hierarchical structure. |
Debiasing Multilingual LLMs in Cross-lingual Latent Space (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have evaluated their cross-lingual transferability by directly applying these methods to LLM representations, revealing their limited effectiveness across languages. |
| Approach: | They propose to perform debiasing in a joint latent space rather than directly on LLM representations by using an autoencoder trained on parallel TED talk scripts. |
| Outcome: | The proposed method improves both the overall debiasing performance and cross-lingual transferability of the proposed techniques across four languages. |
Rejection-to-Acceptance Transition: Model Editing-Based Jailbreak Backdoor Injection Not Limited to Few Output Tokens (2026.findings-acl)
Copied to clipboard
Shiji Yang, Min Cai, Hao Xiong, Congyao Mei, Haodong Zou, Shicheng Tan, Jie Chen, Fulan Qian, Shu Zhao
| Challenge: | Existing methods for jailbreaking LLMs are implemented by binding backdoors to predefined phrases as first few output tokens, inducing the LLM’s next-token prediction to produce continuous responses. |
| Approach: | They propose a model editing-based jailbreak backdoor attack that hijacks LLM representations into a acceptance domain rather than binding to a few output tokens. |
| Outcome: | The proposed model editing method outperforms existing methods, showing stronger jailbreak capabilities across LLMs and datasets. |